Words List (appearance)
# word phonetic sentence
1 LSVRC [!≈ el es vi: ɑ:(r) si:]
  • We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. 我们训练了一个大型深度卷积神经网络来将ImageNet LSVRC-2010竞赛的120万高分辨率的图像分到1000不同的类别中。
2 non-saturating [!≈ nɒn ˈsætʃəreitɪŋ]
  • To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. 为了训练的更快，我们使用了非饱和神经元并对卷积操作进行了非常有效的GPU实现。
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$. 考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
3 variant [ˈveəriənt]
  • We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. 我们也使用这个模型的一个变种参加了ILSVRC-2012竞赛，赢得了冠军并且与第二名 top-5 26.2%的错误率相比，我们取得了top-5 15.3%的错误率。
4 ILSVRC [!≈ aɪ el es vi: ɑ:(r) si:]
  • We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry. 我们也使用这个模型的一个变种参加了ILSVRC-2012竞赛，赢得了冠军并且与第二名 top-5 26.2%的错误率相比，我们取得了top-5 15.3%的错误率。
  • The specific contributions of this paper are as follows: we trained one of the largest convolutional neural networks to date on the subsets of ImageNet used in the ILSVRC-2010 and ILSVRC-2012 competitions [2] and achieved by far the best results ever reported on these datasets. 本文具体的贡献如下：我们在ILSVRC-2010和ILSVRC-2012[2]的ImageNet子集上训练了到目前为止最大的神经网络之一，并取得了迄今为止在这些数据集上报道过的最好结果。
  • The specific contributions of this paper are as follows: we trained one of the largest convolutional neural networks to date on the subsets of ImageNet used in the ILSVRC-2010 and ILSVRC-2012 competitions [2] and achieved by far the best results ever reported on these datasets. 本文具体的贡献如下：我们在ILSVRC-2010和ILSVRC-2012[2]的ImageNet子集上训练了到目前为止最大的神经网络之一，并取得了迄今为止在这些数据集上报道过的最好结果。
  • Starting in 2010, as part of the Pascal Visual Object Challenge, an annual competition called the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) has been held. 从2010年起，作为Pascal视觉对象挑战赛的一部分，每年都会举办ImageNet大规模视觉识别挑战赛（ILSVRC）。
  • ILSVRC uses a subset of ImageNet with roughly 1000 images in each of 1000 categories. ILSVRC使用ImageNet的一个子集，1000个类别每个类别大约1000张图像。
  • ILSVRC-2010 is the only version of ILSVRC for which the test set labels are available, so this is the version on which we performed most of our experiments. ILSVRC-2010是ILSVRC竞赛中唯一可以获得测试集标签的版本，因此我们大多数实验都是在这个版本上运行的。
  • ILSVRC-2010 is the only version of ILSVRC for which the test set labels are available, so this is the version on which we performed most of our experiments. ILSVRC-2010是ILSVRC竞赛中唯一可以获得测试集标签的版本，因此我们大多数实验都是在这个版本上运行的。
  • Since we also entered our model in the ILSVRC-2012 competition, in Section 6 we report our results on this version of the dataset as well, for which test set labels are unavailable. 由于我们也使用我们的模型参加了ILSVRC-2012竞赛，因此在第六节我们也报告了模型在这个版本的数据集上的结果，这个版本的测试标签是不可获得的。
  • Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label, this turns out to be insufficient to learn so many parameters without considerable overfitting. 尽管ILSVRC的1000类使每个训练样本从图像到标签的映射上强加了10比特的约束，但这不足以学习这么多的参数而没有相当大的过拟合。
  • Our results on ILSVRC-2010 are summarized in Table 1. 我们在ILSVRC-2010上的结果概括为表1。
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24]. 在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
  • Table 1: Comparison of results on ILSVRC-2010 test set. 表1：ILSVRC-2010测试集上的结果对比。
  • We also entered our model in the ILSVRC-2012 competition and report our results in Table 2. 我们也用我们的模型参加了ILSVRC-2012竞赛并在表2中报告了我们的结果。
  • Since the ILSVRC-2012 test set labels are not publicly available, we cannot report test error rates for all the models that we tried. 由于ILSVRC-2012的测试集标签不可以公开得到，我们不能报告我们尝试的所有模型的测试错误率。
  • Training one CNN, with an extra sixth convolutional layer over the last pooling layer, to classify the entire ImageNet Fall 2011 release (15M images, 22K categories), and then “fine-tuning” it on ILSVRC-2012 gives an error rate of 16.6%. 为了对ImageNet 2011秋季发布的整个数据集（1500万图像，22000个类别）进行分类，我们在最后的池化层之后有一个额外的第6卷积层，训练了一个CNN，然后在它上面进行“fine-tuning”，在ILSVRC-2012取得了16.6%的错误率。
  • Table 2: Comparison of error rates on ILSVRC-2012 validation and test sets. 表2：ILSVRC-2012验证集和测试集的误差对比。
  • Figure 4: (Left) Eight ILSVRC-2010 test images and the five labels considered most probable by our model. 图4：（左）8张ILSVRC-2010测试图像和我们的模型认为最可能的5个标签。
  • (Right) Five ILSVRC-2010 test images in the first column. （右）第一列是5张ILSVRC-2010测试图像。
5 e.g. [ˌi: ˈdʒi:]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations. 直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • And indeed, the shortcomings of small image datasets have been widely recognized (e.g., Pinto et al. [21]), but it has only recently become possible to collect labeled datasets with millions of images. 实际上，小图像数据集的缺点已经被广泛认识到（例如，Pinto et al. [21]），但收集上百万图像的标注数据仅在最近才变得的可能。
  • Traditionally, the neighborhoods summarized by adjacent pooling units do not overlap (e.g., [17, 11, 4]). 习惯上，相邻池化单元归纳的区域是不重叠的（例如[17, 11, 4]）。
  • The easiest and most common method to reduce overfitting on image data is to artificially enlarge the dataset using label-preserving transformations (e.g., [25, 4, 5]). 图像数据上最简单常用的用来减少过拟合的方法是使用标签保留变换（例如[25, 4, 5]）来人工增大数据集。
6 NORB [!≈ en əu ɑ:(r) bi:]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations. 直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
7 Caltech ['kæltek]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations. 直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • For example, Jarrett et al. [11] claim that the nonlinearity $f(x) = \lvert \tanh(x) \rvert$ works particularly well with their type of contrast normalization followed by local average pooling on the Caltech-101 dataset. 例如，Jarrett等人[11]声称非线性函数$f(x) = \lvert \tanh(x) \rvert$与其对比度归一化一起，然后是局部均值池化，在Caltech-101数据集上工作的非常好。
8 CIFAR [!≈ si: aɪ ef eɪ ɑ:(r)]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations. 直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • This is demonstrated in Figure 1, which shows the number of iterations required to reach 25% training error on the CIFAR-10 dataset for a particular four-layer convolutional network. 在图1中，对于一个特定的四层卷积网络，在CIFAR-10数据集上达到25%的训练误差所需要的迭代次数可以证实这一点。
  • Figure 1: A four-layer convolutional neural network with ReLUs (solid line) reaches a 25% training error rate on CIFAR-10 six times faster than an equivalent network with tanh neurons (dashed line). 图1：使用ReLU的四层卷积神经网络在CIFAR-10数据集上达到25%的训练误差比使用tanh神经元的等价网络（虚线）快六倍。
  • We also verified the effectiveness of this scheme on the CIFAR-10 dataset: a four-layer CNN achieved a 13% test error rate without normalization and 11% with normalization. 我们也在CIFAR-10数据集上验证了这个方案的有效性：一个乜嘢归一化的四层CNN取得了13%的错误率，而使用归一化取得了11%的错误率。
9 augment [ɔ:gˈment]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations. 直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
10 label-preserving [!≈ ˈleɪbl prɪ'zɜ:vɪŋ]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations. 直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • The easiest and most common method to reduce overfitting on image data is to artificially enlarge the dataset using label-preserving transformations (e.g., [25, 4, 5]). 图像数据上最简单常用的用来减少过拟合的方法是使用标签保留变换（例如[25, 4, 5]）来人工增大数据集。
11 MNIST [!≈ em en aɪ es ti:]
  • For example, the current best error rate on the MNIST digit-recognition task (<0.3%) approaches human performance [4]. 例如，目前在MNIST数字识别任务上（<0.3%）的最好准确率已经接近了人类水平[4]。
12 variability [ˌveəriəˈbɪləti]
  • But objects in realistic settings exhibit considerable variability, so to learn to recognize them it is necessary to use much larger training sets. 但真实环境中的对象表现出了相当大的可变性，因此为了学习识别它们，有必要使用更大的训练数据集。
13 Pinto [ˈpɪntəʊ]
  • And indeed, the shortcomings of small image datasets have been widely recognized (e.g., Pinto et al. [21]), but it has only recently become possible to collect labeled datasets with millions of images. 实际上，小图像数据集的缺点已经被广泛认识到（例如，Pinto et al. [21]），但收集上百万图像的标注数据仅在最近才变得的可能。
14 LabelMe
  • The new larger datasets include LabelMe [23], which consists of hundreds of thousands of fully-segmented images, and ImageNet [6], which consists of over 15 million labeled high-resolution images in over 22,000 categories. 新的更大的数据集包括LabelMe [23]，它包含了数十万张完全分割的图像，ImageNet[6]，它包含了22000个类别上的超过1500万张标注的高分辨率的图像。
15 immense [ɪˈmens]
  • However, the immense complexity of the object recognition task means that this problem cannot be specified even by a dataset as large as ImageNet, so our model should also have lots of prior knowledge to compensate for all the data we don’t have. 然而对象识别任务的巨大复杂性意味着这个问题不能被指定，即使通过像ImageNet这样的大数据集，因此我们的模型应该也有许多先验知识来补偿我们所没有的数据。
16 constitute [ˈkɒnstɪtju:t]
  • Convolutional neural networks (CNNs) constitute one such class of models [16, 11, 13, 18, 15, 22, 26]. 卷积神经网络(CNNs)构成了一个这样的模型[16, 11, 13, 18, 15, 22, 26]。
17 stationarity [steɪʃə'nærɪtɪ]
  • Their capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies). 它们的能力可以通过改变它们的广度和深度来控制，它们也可以对图像的本质进行强大且通常正确的假设（也就是说，统计的稳定性和像素依赖的局部性）。
18 feedforward [fi:d'fɔ:wəd]
  • Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse. 因此，与具有层次大小相似的标准前馈神经网络，CNNs有更少的连接和参数，因此它们更容易训练，而它们理论上的最佳性能可能仅比标准前馈神经网络差一点。
19 similarly-sized [!≈ ˈsɪmələli 'saɪzd]
  • Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse. 因此，与具有层次大小相似的标准前馈神经网络，CNNs有更少的连接和参数，因此它们更容易训练，而它们理论上的最佳性能可能仅比标准前馈神经网络差一点。
20 theoretically-best [!≈ ˌθɪə'retɪklɪ best]
  • Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse. 因此，与具有层次大小相似的标准前馈神经网络，CNNs有更少的连接和参数，因此它们更容易训练，而它们理论上的最佳性能可能仅比标准前馈神经网络差一点。
21 prohibitively [prəʊˈhɪbɪtɪvlɪ]
  • Despite the attractive qualities of CNNs, and despite the relative efficiency of their local architecture, they have still been prohibitively expensive to apply in large scale to high-resolution images. 尽管CNN具有引人注目的特性，尽管CNN的局部架构相当有效，但将它们大规模的应用到到高分辨率图像中仍然是代价极高的。
22 interestingly-large [!≈ 'intristiŋli lɑ:dʒ]
  • Luckily, current GPUs, paired with a highly-optimized implementation of 2D convolution, are powerful enough to facilitate the training of interestingly-large CNNs, and recent datasets such as ImageNet contain enough labeled examples to train such models without severe overfitting. 幸运的是，目前的GPU，搭配了高度优化的2D卷积实现，强大到足够促进有趣地大量CNN的训练，最近的数据集例如ImageNet包含足够的标注样本来训练这样的模型而没有严重的过拟合。
23 GTX [!≈ dʒi: ti: eks]
  • Our network takes between five and six days to train on two GTX 580 3GB GPUs. 我们的网络在两个GTX 580 3GB GPU上训练五六天。
  • A single GTX 580 GPU has only 3GB of memory, which limits the maximum size of the networks that can be trained on it. 单个GTX580 GPU只有3G内存，这限制了可以在GTX580上进行训练的网络最大尺寸。
  • We trained the network for roughly 90 cycles through the training set of 1.2 million images, which took five to six days on two NVIDIA GTX 580 3GB GPUs. 我们在120万图像的训练数据集上训练神经网络大约90个循环，在两个NVIDIA GTX 580 3GB GPU上花费了五到六天。
24 labelers [!≈ 'leɪblərs]
  • The images were collected from the web and labeled by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool. 这些图像是从网上收集的，使用了Amazon’s Mechanical Turk的众包工具通过人工标注的。
25 Turk [tә:k]
  • The images were collected from the web and labeled by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool. 这些图像是从网上收集的，使用了Amazon’s Mechanical Turk的众包工具通过人工标注的。
26 crowd-sourcing [!≈ kraʊd 'saʊəsɪŋ]
  • The images were collected from the web and labeled by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool. 这些图像是从网上收集的，使用了Amazon’s Mechanical Turk的众包工具通过人工标注的。
27 Pascal ['pæskәl]
  • Starting in 2010, as part of the Pascal Visual Object Challenge, an annual competition called the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) has been held. 从2010年起，作为Pascal视觉对象挑战赛的一部分，每年都会举办ImageNet大规模视觉识别挑战赛（ILSVRC）。
28 variable-resolution [!≈ ˈveəriəbl ˌrezəˈlu:ʃn]
  • ImageNet consists of variable-resolution images, while our system requires a constant input dimensionality. ImageNet包含各种分辨率的图像，而我们的系统要求不变的输入维度。
29 down-sampled [!≈ daʊn 'sɑ:mpld]
  • Therefore, we down-sampled the images to a fixed resolution of 256 × 256. 因此，我们将图像进行下采样到固定的256×256分辨率。
30 rectangular [rek'tæŋɡjələ(r)]
  • Given a rectangular image, we first rescaled the image such that the shorter side was of length 256, and then cropped out the central 256×256 patch from the resulting image. 给定一个矩形图像，我们首先缩放图像短边长度为256，然后从结果图像中裁剪中心的256×256大小的图像块。
31 rescale [ri:'skeɪl]
  • Given a rectangular image, we first rescaled the image such that the shorter side was of length 256, and then cropped out the central 256×256 patch from the resulting image. 给定一个矩形图像，我们首先缩放图像短边长度为256，然后从结果图像中裁剪中心的256×256大小的图像块。
32 nonlinearity [nɒnlɪnɪ'ærɪtɪ]
  • 3.1 ReLU Nonlinearity 3.1 ReLU非线性
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$. 考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$. 考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs). 根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
  • For example, Jarrett et al. [11] claim that the nonlinearity $f(x) = \lvert \tanh(x) \rvert$ works particularly well with their type of contrast normalization followed by local average pooling on the Caltech-101 dataset. 例如，Jarrett等人[11]声称非线性函数$f(x) = \lvert \tanh(x) \rvert$与其对比度归一化一起，然后是局部均值池化，在Caltech-101数据集上工作的非常好。
  • Denoting by $a_{x,y}^i$ the activity of a neuron computed by applying kernel $i$ at position $(x, y)$ and then applying the ReLU nonlinearity, the response-normalized activity $b^i_{x,y}$ is given by the expression $a_{x,y}^i$表示神经元激活，通过在$(x, y)$位置应用核i，然后应用ReLU非线性来计算，响应归一化激活$b^i_{x,y}$通过下式给定：
  • We applied this normalization after applying the ReLU nonlinearity in certain layers (see Section 3.5). 我们在特定的层使用的ReLU非线性之后应用了这种归一化（请看3.5小节）。
33 descent [dɪˈsent]
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$. 考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • We trained our models using stochastic gradient descent with a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005. 我们使用随机梯度下降来训练我们的模型，样本的batch size为128，动量为0.9，权重衰减为0.0005。
34 saturate [ˈsætʃəreɪt]
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$. 考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • This plot shows that we would not have been able to experiment with such large neural networks for this work if we had used traditional saturating neuron models. 这幅图表明，如果我们采用传统的饱和神经元模型，我们将不能在如此大的神经网络上实验该工作。
  • The magnitude of the effect demonstrated here varies with network architecture, but networks with ReLUs consistently learn several times faster than equivalents with saturating neurons. 影响的大小随着网络结构的变化而变化，这一点已得到证实，但使用ReLU的网络都比等价的饱和神经元快几倍。
  • ReLUs have the desirable property that they do not require input normalization to prevent them from saturating. ReLU具有让人满意的特性，它不需要通过输入归一化来防止饱和。
35 Nair ['nɑ:ɪə]
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs). 根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
36 Hinton
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs). 根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
37 rectify [ˈrektɪfaɪ]
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs). 根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
38 dash [dæʃ]
  • Figure 1: A four-layer convolutional neural network with ReLUs (solid line) reaches a 25% training error rate on CIFAR-10 six times faster than an equivalent network with tanh neurons (dashed line). 图1：使用ReLU的四层卷积神经网络在CIFAR-10数据集上达到25%的训练误差比使用tanh神经元的等价网络（虚线）快六倍。
39 consistently [kən'sɪstəntlɪ]
  • The magnitude of the effect demonstrated here varies with network architecture, but networks with ReLUs consistently learn several times faster than equivalents with saturating neurons. 影响的大小随着网络结构的变化而变化，这一点已得到证实，但使用ReLU的网络都比等价的饱和神经元快几倍。
40 Jarrett
  • For example, Jarrett et al. [11] claim that the nonlinearity $f(x) = \lvert \tanh(x) \rvert$ works particularly well with their type of contrast normalization followed by local average pooling on the Caltech-101 dataset. 例如，Jarrett等人[11]声称非线性函数$f(x) = \lvert \tanh(x) \rvert$与其对比度归一化一起，然后是局部均值池化，在Caltech-101数据集上工作的非常好。
  • This scheme bears some resemblance to the local contrast normalization scheme of Jarrett et al. [11], but ours would be more correctly termed “brightness normalization”, since we do not subtract the mean activity. 这个方案与Jarrett等人[11]的局部对比度归一化方案有一定的相似性，但我们更恰当的称其为“亮度归一化”，因此我们没有减去均值。
41 cross-GPU
  • Current GPUs are particularly well-suited to cross-GPU parallelization, as they are able to read from and write to one another’s memory directly, without going through host machine memory. 目前的GPU非常适合跨GPU并行，因为它们可以直接互相读写内存，而不需要通过主机内存。
42 parallelization [pærəlɪlaɪ'zeɪʃn]
  • Current GPUs are particularly well-suited to cross-GPU parallelization, as they are able to read from and write to one another’s memory directly, without going through host machine memory. 目前的GPU非常适合跨GPU并行，因为它们可以直接互相读写内存，而不需要通过主机内存。
  • The parallelization scheme that we employ essentially puts half of the kernels (or neurons) on each GPU, with one additional trick: the GPUs communicate only in certain layers. 我们采用的并行方案基本上每个GPU放置一半的核（或神经元），还有一个额外的技巧：只在某些特定的层上进行GPU通信。
43 connectivity [ˌkɒnekˈtɪvɪti]
  • Choosing the pattern of connectivity is a problem for cross-validation, but this allows us to precisely tune the amount of communication until it is an acceptable fraction of the amount of computation. 连接模式的选择是一个交叉验证问题，但这可以让我们准确地调整通信数量，直到它的计算量在可接受的范围内。
  • Notice the specialization exhibited by the two GPUs, a result of the restricted connectivity described in Section 3.5. 注意两个GPU表现出的专业化，3.5小节中描述的受限连接的结果。
44 cross-validation [k'rɒsvælɪd'eɪʃn]
  • Choosing the pattern of connectivity is a problem for cross-validation, but this allows us to precisely tune the amount of communication until it is an acceptable fraction of the amount of computation. 连接模式的选择是一个交叉验证问题，但这可以让我们准确地调整通信数量，直到它的计算量在可接受的范围内。
45 resultant [rɪˈzʌltənt]
  • The resultant architecture is somewhat similar to that of the “columnar” CNN employed by Ciresan et al. [5], except that our columns are not independent (see Figure 2). 除了我们的列不是独立的之外（看图2），最终的架构有点类似于Ciresan等人[5]采用的“columnar” CNN。
46 columnar [kə'lʌmnə]
  • The resultant architecture is somewhat similar to that of the “columnar” CNN employed by Ciresan et al. [5], except that our columns are not independent (see Figure 2). 除了我们的列不是独立的之外（看图2），最终的架构有点类似于Ciresan等人[5]采用的“columnar” CNN。
47 Ciresan
  • The resultant architecture is somewhat similar to that of the “columnar” CNN employed by Ciresan et al. [5], except that our columns are not independent (see Figure 2). 除了我们的列不是独立的之外（看图2），最终的架构有点类似于Ciresan等人[5]采用的“columnar” CNN。
48 delineation [dɪˌlɪnɪ'eɪʃn]
  • Figure 2: An illustration of the architecture of our CNN, explicitly showing the delineation of responsibilities between the two GPUs. 图 2：我们CNN架构图解，明确描述了两个GPU之间的责任。
49 layer-part [!≈ ˈleɪə(r) pɑ:t]
  • One GPU runs the layer-parts at the top of the figure while the other runs the layer-parts at the bottom. 在图的顶部，一个GPU运行在部分层上，而在图的底部，另一个GPU运行在部分层上。
  • One GPU runs the layer-parts at the top of the figure while the other runs the layer-parts at the bottom. 在图的顶部，一个GPU运行在部分层上，而在图的底部，另一个GPU运行在部分层上。
50 response-normalized [!≈ rɪˈspɒns 'nɔ:məlaɪzd]
  • Denoting by $a_{x,y}^i$ the activity of a neuron computed by applying kernel $i$ at position $(x, y)$ and then applying the ReLU nonlinearity, the response-normalized activity $b^i_{x,y}$ is given by the expression $a_{x,y}^i$表示神经元激活，通过在$(x, y)$位置应用核i，然后应用ReLU非线性来计算，响应归一化激活$b^i_{x,y}$通过下式给定：
  • The second convolutional layer takes as input the (response-normalized and pooled) output of the first convolutional layer and filters it with 256 kernels of size 5 × 5 × 48. 第2卷积层使用用第1卷积层的输出（响应归一化和池化）作为输入，并使用256个核进行滤波，核大小为5 × 5 × 48。
51 arbitrary [ˈɑ:bɪtrəri]
  • The ordering of the kernel maps is of course arbitrary and determined before training begins. 核映射的顺序当然是任意的，在训练开始前确定。
52 lateral [ˈlætərəl]
  • This sort of response normalization implements a form of lateral inhibition inspired by the type found in real neurons, creating competition for big activities amongst neuron outputs computed using different kernels. 响应归一化的顺序实现了一种侧抑制形式，灵感来自于真实神经元中发现的类型，为使用不同核进行神经元输出计算的较大活动创造了竞争。
53 inhibition [ˌɪnhɪˈbɪʃn]
  • This sort of response normalization implements a form of lateral inhibition inspired by the type found in real neurons, creating competition for big activities amongst neuron outputs computed using different kernels. 响应归一化的顺序实现了一种侧抑制形式，灵感来自于真实神经元中发现的类型，为使用不同核进行神经元输出计算的较大活动创造了竞争。
54 brightness ['braɪtnəs]
  • This scheme bears some resemblance to the local contrast normalization scheme of Jarrett et al. [11], but ours would be more correctly termed “brightness normalization”, since we do not subtract the mean activity. 这个方案与Jarrett等人[11]的局部对比度归一化方案有一定的相似性，但我们更恰当的称其为“亮度归一化”，因此我们没有减去均值。
55 neighboring ['neɪbərɪŋ]
  • Pooling layers in CNNs summarize the outputs of neighboring groups of neurons in the same kernel map. CNN中的池化层归纳了同一核映射上相邻组神经元的输出。
  • The first convolutional layer filters the 224 × 224 × 3 input image with 96 kernels of size 11 × 11 × 3 with a stride of 4 pixels (this is the distance between the receptive field centers of neighboring neurons in a kernel map). 第1卷积层使用96个核对224 × 224 × 3的输入图像进行滤波，核大小为11 × 11 × 3，步长是4个像素（核映射中相邻神经元感受野中心之间的距离）。
56 overfit
  • We generally observe during training that models with overlapping pooling find it slightly more difficult to overfit. 我们在训练过程中通常观察采用重叠池化的模型，发现它更难过拟合。
57 multinomial [ˌmʌltɪ'nəʊmɪəl]
  • Our network maximizes the multinomial logistic regression objective, which is equivalent to maximizing the average across training cases of the log-probability of the correct label under the prediction distribution. 我们的网络最大化多项逻辑回归的目标，这等价于最大化预测分布下训练样本正确标签的对数概率的均值。
58 log-probability [!≈ lɒg ˌprɒbəˈbɪləti]
  • Our network maximizes the multinomial logistic regression objective, which is equivalent to maximizing the average across training cases of the log-probability of the correct label under the prediction distribution. 我们的网络最大化多项逻辑回归的目标，这等价于最大化预测分布下训练样本正确标签的对数概率的均值。
59 response-normalization [!≈ rɪˈspɒns ˌnɔ:məlaɪ'zeɪʃn]
  • Response-normalization layers follow the first and second convolutional layers. 第1，2卷积层之后是响应归一化层。
  • Max-pooling layers, of the kind described in Section 3.4, follow both response-normalization layers as well as the fifth convolutional layer. 3.4节描述的这种最大池化层在响应归一化层和第5卷积层之后。
60 non-linearity ['nɒnlaɪn'ərɪtɪ]
  • The ReLU non-linearity is applied to the output of every convolutional and fully-connected layer. ReLU非线性应用在每个卷积层和全连接层的输出上。
61 receptive [rɪˈseptɪv]
  • The first convolutional layer filters the 224 × 224 × 3 input image with 96 kernels of size 11 × 11 × 3 with a stride of 4 pixels (this is the distance between the receptive field centers of neighboring neurons in a kernel map). 第1卷积层使用96个核对224 × 224 × 3的输入图像进行滤波，核大小为11 × 11 × 3，步长是4个像素（核映射中相邻神经元感受野中心之间的距离）。
62 normalize [ˈnɔ:məlaɪz]
  • The third convolutional layer has 384 kernels of size 3 × 3 × 256 connected to the (normalized, pooled) outputs of the second convolutional layer. 第3卷积层有384个核，核大小为3 × 3 × 256，与第2卷积层的输出（归一化的，池化的）相连。
63 bits ['bɪts]
  • Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label, this turns out to be insufficient to learn so many parameters without considerable overfitting. 尽管ILSVRC的1000类使每个训练样本从图像到标签的映射上强加了10比特的约束，但这不足以学习这么多的参数而没有相当大的过拟合。
64 augmentation [ˌɔ:ɡmen'teɪʃn]
  • 4.1 Data Augmentation 4.1 数据增强
  • We employ two distinct forms of data augmentation, both of which allow transformed images to be produced from the original images with very little computation, so the transformed images do not need to be stored on disk. 我们使用了两种独特的数据增强方式，这两种方式都可以从原始图像通过非常少的计算量产生变换的图像，因此变换图像不需要存储在硬盘上。
  • So these data augmentation schemes are, in effect, computationally free. 因此，实际上这些数据增强方案是计算免费的。
  • The first form of data augmentation consists of generating image translations and horizontal reflections. 第一种数据增强方式包括产生图像变换和水平翻转。
  • The second form of data augmentation consists of altering the intensities of the RGB channels in training images. 第二种数据增强方式包括改变训练图像的RGB通道的强度。
65 artificially [ˌɑ:tɪ'fɪʃəlɪ]
  • The easiest and most common method to reduce overfitting on image data is to artificially enlarge the dataset using label-preserving transformations (e.g., [25, 4, 5]). 图像数据上最简单常用的用来减少过拟合的方法是使用标签保留变换（例如[25, 4, 5]）来人工增大数据集。
66 computationally [!≈ ˌkɒmpjuˈteɪʃənli]
  • So these data augmentation schemes are, in effect, computationally free. 因此，实际上这些数据增强方案是计算免费的。
67 interdependent [ˌɪntədɪˈpendənt]
  • This increases the size of our training set by a factor of 2048, though the resulting training examples are, of course, highly interdependent. 这通过一个2048因子增大了我们的训练集，尽管最终的训练样本是高度相关的。
68 PCA [!≈ pi: si: eɪ]
  • Specifically, we perform PCA on the set of RGB pixel values throughout the ImageNet training set. 具体地，我们在整个ImageNet训练集上对RGB像素值集合执行PCA。
69 proportional [prəˈpɔ:ʃənl]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1. 对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
70 eigenvalue ['aɪdʒənˌvælju:]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1. 对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable. $p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
71 Gaussian ['gaʊsɪən]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1. 对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
  • We initialized the weights in each layer from a zero-mean Gaussian distribution with standard deviation 0.01. 我们使用均值为0，标准差为0.01的高斯分布对每一层的权重进行初始化。
72 deviation [ˌdi:viˈeɪʃn]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1. 对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
  • We initialized the weights in each layer from a zero-mean Gaussian distribution with standard deviation 0.01. 我们使用均值为0，标准差为0.01的高斯分布对每一层的权重进行初始化。
73 ith
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable. $p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
  • where $i$ is the iteration index, $v$ is the momentum variable, $\varepsilon$ is the learning rate, and $\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$ is the average over the ith batch $D_i$ of the derivative of the objective with respect to $w$, evaluated at $w_i$. $i$是迭代索引，$v$是动量变量，$\varepsilon$是学习率，$\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$是目标函数对$w$，在$w_i$上的第$i$批微分$D_i$的平均。
74 eigenvector ['aɪgənvektə(r)]
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable. $p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
75 covariance [kəʊ'veərɪəns]
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable. $p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
76 aforementioned [əˌfɔ:ˈmenʃənd]
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable. $p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
  • Averaging the predictions of two CNNs that were pre-trained on the entire Fall 2011 release with the aforementioned five CNNs gives an error rate of 15.3%. 对在ImageNet 2011秋季发布的整个数据集上预训练的两个CNN和前面提到的五个CNN的预测进行平均得到了15.3%的错误率。
77 invariant [ɪnˈveəriənt]
  • This scheme approximately captures an important property of natural images, namely, that object identity is invariant to changes in the intensity and color of the illumination. 这个方案近似抓住了自然图像的一个重要特性，即光照的颜色和强度发生变化时，目标身份是不变的。
78 co-adaptation [!≈ kəʊ ˌædæpˈteɪʃn]
  • This technique reduces complex co-adaptations of neurons, since a neuron cannot rely on the presence of particular other neurons. 这个技术减少了复杂的神经元互适应，因为一个神经元不能依赖特定的其它神经元的存在。
79 conjunction [kənˈdʒʌŋkʃn]
  • It is, therefore, forced to learn more robust features that are useful in conjunction with many different random subsets of the other neurons. 因此，神经元被强迫学习更鲁棒的特征，它在与许多不同的其它神经元的随机子集结合时是有用的。
80 approximation [əˌprɒksɪˈmeɪʃn]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks. 在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
81 geometric [ˌdʒi:əˈmetrɪk]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks. 在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
82 predictive [prɪˈdɪktɪv]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks. 在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
83 exponentially-many [!≈ ˌekspə'nenʃəlɪ ˈmeni]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks. 在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
84 stochastic [stə'kæstɪk]
  • We trained our models using stochastic gradient descent with a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005. 我们使用随机梯度下降来训练我们的模型，样本的batch size为128，动量为0.9，权重衰减为0.0005。
85 momentum [məˈmentəm]
  • We trained our models using stochastic gradient descent with a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005. 我们使用随机梯度下降来训练我们的模型，样本的batch size为128，动量为0.9，权重衰减为0.0005。
  • where $i$ is the iteration index, $v$ is the momentum variable, $\varepsilon$ is the learning rate, and $\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$ is the average over the ith batch $D_i$ of the derivative of the objective with respect to $w$, evaluated at $w_i$. $i$是迭代索引，$v$是动量变量，$\varepsilon$是学习率，$\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$是目标函数对$w$，在$w_i$上的第$i$批微分$D_i$的平均。
86 regularizer
  • In other words, weight decay here is not merely a regularizer: it reduces the model’s training error. 换句话说，权重衰减不仅仅是一个正则项：它减少了模型的训练误差。
87 derivative [dɪˈrɪvətɪv]
  • where $i$ is the iteration index, $v$ is the momentum variable, $\varepsilon$ is the learning rate, and $\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$ is the average over the ith batch $D_i$ of the derivative of the objective with respect to $w$, evaluated at $w_i$. $i$是迭代索引，$v$是动量变量，$\varepsilon$是学习率，$\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$是目标函数对$w$，在$w_i$上的第$i$批微分$D_i$的平均。
88 heuristic [hjuˈrɪstɪk]
  • The heuristic which we followed was to divide the learning rate by 10 when the validation error rate stopped improving with the current learning rate. 当验证误差在当前的学习率下停止提供时，我们遵循启发式的方法将学习率除以10。
89 termination [ˌtɜ:mɪˈneɪʃn]
  • The learning rate was initialized at 0.01 and reduced three times prior to termination. 学习率初始化为0.01，在训练停止之前降低三次。
90 NVIDIA [ɪn'vɪdɪə]
  • We trained the network for roughly 90 cycles through the training set of 1.2 million images, which took five to six days on two NVIDIA GTX 580 3GB GPUs. 我们在120万图像的训练数据集上训练神经网络大约90个循环，在两个NVIDIA GTX 580 3GB GPU上花费了五到六天。
91 sparse-coding [!≈ spɑ:s 'kəʊdɪŋ]
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24]. 在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
92 FVs
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24]. 在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
  • The second-best contest entry achieved an error rate of 26.2% with an approach that averages the predictions of several classifiers trained on FVs computed from different types of densely-sampled features [7]. 第二名的最好竞赛输入取得了26.2%的错误率，他的方法是对FV上训练的一些分类器的预测结果进行平均，FV在不同类型密集采样特征计算得到的。
93 densely-sampled [!≈ denslɪ 'sɑ:mpld]
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24]. 在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
  • The second-best contest entry achieved an error rate of 26.2% with an approach that averages the predictions of several classifiers trained on FVs computed from different types of densely-sampled features [7]. 第二名的最好竞赛输入取得了26.2%的错误率，他的方法是对FV上训练的一些分类器的预测结果进行平均，FV在不同类型密集采样特征计算得到的。
94 italic [ɪˈtælɪk]
  • In italics are best results achieved by others. 斜体是其它人取得的最好结果。
  • In italics are best results achieved by others. 斜线部分是其它人取得的最好的结果。
95 interchangeably [ɪntəˈtʃeɪndʒəblɪ]
  • In the remainder of this paragraph, we use validation and test error rates interchangeably because in our experience they do not differ by more than 0.1% (see Table 2). 在这段的其余部分，我们会使用验证误差率和测试误差率互换，因为在我们的实验中它们的差别不会超过0.1%（看图2）。
96 asterisk [ˈæstərɪsk]
  • Models with an asterisk were “pre-trained” to classify the entire ImageNet 2011 Fall release. 带星号的是“预训练的”对ImageNet 2011秋季数据集进行分类的模型。
97 convention [kənˈvenʃn]
  • On this dataset we follow the convention in the literature of using half of the images for training and half for testing. 在这个数据集上我们按照惯例用一半的图像来训练，一半的图像来测试。
98 appreciably [ə'pri:ʃəblɪ]
  • Since there is no established test set, our split necessarily differs from the splits used by previous authors, but this does not affect the results appreciably. 由于没有建立测试集，我们的数据集分割有必要不同于以前作者的数据集分割，但这对结果没有明显的影响。
99 Qualitative [ˈkwɒlɪtətɪv]
  • 6.1 Qualitative Evaluations 6.1 定性评估
100 data-connected [!≈ ˈdeɪtə kə'nektɪd]
  • Figure 3 shows the convolutional kernels learned by the network’s two data-connected layers. 图3显示了网络的两个数据连接层学习到的卷积核。
101 orientation-selective [!≈ ˌɔ:riənˈteɪʃn sɪˈlektɪv]
  • The network has learned a variety of frequency and orientation-selective kernels, as well as various colored blobs. 网络学习到了大量的频率核、方向选择核，也学到了各种颜色点。
102 blob [blɒb]
  • The network has learned a variety of frequency and orientation-selective kernels, as well as various colored blobs. 网络学习到了大量的频率核、方向选择核，也学到了各种颜色点。
103 specialization [ˌspeʃəlaɪ'zeɪʃn]
  • Notice the specialization exhibited by the two GPUs, a result of the restricted connectivity described in Section 3.5. 注意两个GPU表现出的专业化，3.5小节中描述的受限连接的结果。
  • This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs). 这种专业化在每次运行时都会发生，并且是与任何特别的随机权重初始化（以GPU的重新编号为模）无关的。
104 color-agnostic [!≈ 'kʌlə(r) ægˈnɒstɪk]
  • The kernels on GPU 1 are largely color-agnostic, while the kernels on on GPU 2 are largely color-specific. GPU 1上的核主要是没有颜色的，而GPU 2上的核主要是针对颜色的。
105 color-specific [!≈ 'kʌlə(r) spəˈsɪfɪk]
  • The kernels on GPU 1 are largely color-agnostic, while the kernels on on GPU 2 are largely color-specific. GPU 1上的核主要是没有颜色的，而GPU 2上的核主要是针对颜色的。
106 modulo ['mɒdjʊləʊ]
  • This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs). 这种专业化在每次运行时都会发生，并且是与任何特别的随机权重初始化（以GPU的重新编号为模）无关的。
107 renumber ['ri:'nʌmbə]
  • This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs). 这种专业化在每次运行时都会发生，并且是与任何特别的随机权重初始化（以GPU的重新编号为模）无关的。
108 qualitatively ['kwɒlɪtətɪvlɪ]
  • In the left panel of Figure 4 we qualitatively assess what the network has learned by computing its top-5 predictions on eight test images. 在图4的左边部分，我们通过在8张测试图像上计算它的top-5预测定性地评估了网络学习到的东西。
109 off-center ['ɔ:fs'entə]
  • Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net. 注意即使是不在图像中心的目标也能被网络识别，例如左上角的小虫。
110 mite [maɪt]
  • Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net. 注意即使是不在图像中心的目标也能被网络识别，例如左上角的小虫。
111 top-left [!≈ tɒp left]
  • Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net. 注意即使是不在图像中心的目标也能被网络识别，例如左上角的小虫。
112 plausible [ˈplɔ:zəbl]
  • For example, only other types of cat are considered plausible labels for the leopard. 例如，对于美洲豹来说，只有其它类型的猫被认为是看似合理的标签。
113 leopard [ˈlepəd]
  • For example, only other types of cat are considered plausible labels for the leopard. 例如，对于美洲豹来说，只有其它类型的猫被认为是看似合理的标签。
114 grille [ɡrɪl]
  • In some cases (grille, cherry) there is genuine ambiguity about the intended focus of the photograph. 在某些案例（格栅，樱桃）中，网络在意的图片焦点真的很含糊。
115 Euclidean [ju:ˈklidiən]
  • The remaining columns show the six training images that produce feature vectors in the last hidden layer with the smallest Euclidean distance from the feature vector for the test image. 剩下的列展示了6张训练图像，这些图像在最后的隐藏层的特征向量与测试图像的特征向量有最小的欧氏距离。
  • If two images produce feature activation vectors with a small Euclidean separation, we can say that the higher levels of the neural network consider them to be similar. 如果两幅图像生成的特征激活向量之间有较小的欧式距离，我们可以认为神经网络的更高层特征认为它们是相似的。
  • Computing similarity by using Euclidean distance between two 4096-dimensional, real-valued vectors is inefficient, but it could be made efficient by training an auto-encoder to compress these vectors to short binary codes. 通过两个4096维实值向量间的欧氏距离来计算相似性是效率低下的，但通过训练一个自动编码器将这些向量压缩为短二值编码可以使其变得高效。
116 probe [prəʊb]
  • Another way to probe the network’s visual knowledge is to consider the feature activations induced by an image at the last, 4096-dimensional hidden layer. 探索网络可视化知识的另一种方式是思考最后的4096维隐藏层在图像上得到的特征激活。
117 induce [ɪnˈdju:s]
  • Another way to probe the network’s visual knowledge is to consider the feature activations induced by an image at the last, 4096-dimensional hidden layer. 探索网络可视化知识的另一种方式是思考最后的4096维隐藏层在图像上得到的特征激活。
118 retrieve [rɪˈtri:v]
  • Notice that at the pixel level, the retrieved training images are generally not close in L2 to the query images in the first column. 注意在像素级别，检索到的训练图像与第一列的查询图像在L2上通常是不接近的。
  • For example, the retrieved dogs and elephants appear in a variety of poses. 例如，检索的狗和大象似乎有很多姿态。
  • This should produce a much better image retrieval method than applying auto-encoders to the raw pixels [14], which does not make use of image labels and hence has a tendency to retrieve images with similar patterns of edges, whether or not they are semantically similar. 这应该会产生一种比将自动编码器应用到原始像素上[14]更好的图像检索方法，自动编码器应用到原始像素上的方法没有使用图像标签，因此会趋向于检索与要检索的图像具有相似边缘模式的图像，无论它们是否是语义上相似。
119 supplementary [ˌsʌplɪˈmentri]
  • We present the results for many more test images in the supplementary material. 我们在补充材料中对更多的测试图像呈现了这种结果。
120 real-valued ['reɪɑ:lv'ælju:d]
  • Computing similarity by using Euclidean distance between two 4096-dimensional, real-valued vectors is inefficient, but it could be made efficient by training an auto-encoder to compress these vectors to short binary codes. 通过两个4096维实值向量间的欧氏距离来计算相似性是效率低下的，但通过训练一个自动编码器将这些向量压缩为短二值编码可以使其变得高效。
121 retrieval [rɪˈtri:vl]
  • This should produce a much better image retrieval method than applying auto-encoders to the raw pixels [14], which does not make use of image labels and hence has a tendency to retrieve images with similar patterns of edges, whether or not they are semantically similar. 这应该会产生一种比将自动编码器应用到原始像素上[14]更好的图像检索方法，自动编码器应用到原始像素上的方法没有使用图像标签，因此会趋向于检索与要检索的图像具有相似边缘模式的图像，无论它们是否是语义上相似。
122 semantically [sɪ'mæntɪklɪ]
  • This should produce a much better image retrieval method than applying auto-encoders to the raw pixels [14], which does not make use of image labels and hence has a tendency to retrieve images with similar patterns of edges, whether or not they are semantically similar. 这应该会产生一种比将自动编码器应用到原始像素上[14]更好的图像检索方法，自动编码器应用到原始像素上的方法没有使用图像标签，因此会趋向于检索与要检索的图像具有相似边缘模式的图像，无论它们是否是语义上相似。
123 infero-temporal
  • Thus far, our results have improved as we have made our network larger and trained it longer but we still have many orders of magnitude to go in order to match the infero-temporal pathway of the human visual system. 到目前为止，我们的结果已经提高了，因为我们的网络更大、训练时间更长，但为了匹配人类视觉系统的下颞线（视觉专业术语）我们仍然有许多数量级要达到。
124 pathway [ˈpɑ:θweɪ]
  • Thus far, our results have improved as we have made our network larger and trained it longer but we still have many orders of magnitude to go in order to match the infero-temporal pathway of the human visual system. 到目前为止，我们的结果已经提高了，因为我们的网络更大、训练时间更长，但为了匹配人类视觉系统的下颞线（视觉专业术语）我们仍然有许多数量级要达到。

Words List (frequency)
# word (frequency) phonetic sentence
1 ILSVRC
(18) 		[!≈ aɪ el es vi: ɑ:(r) si:]
  • We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.我们也使用这个模型的一个变种参加了ILSVRC-2012竞赛，赢得了冠军并且与第二名 top-5 26.2%的错误率相比，我们取得了top-5 15.3%的错误率。
  • The specific contributions of this paper are as follows: we trained one of the largest convolutional neural networks to date on the subsets of ImageNet used in the ILSVRC-2010 and ILSVRC-2012 competitions [2] and achieved by far the best results ever reported on these datasets.本文具体的贡献如下：我们在ILSVRC-2010和ILSVRC-2012[2]的ImageNet子集上训练了到目前为止最大的神经网络之一，并取得了迄今为止在这些数据集上报道过的最好结果。
  • The specific contributions of this paper are as follows: we trained one of the largest convolutional neural networks to date on the subsets of ImageNet used in the ILSVRC-2010 and ILSVRC-2012 competitions [2] and achieved by far the best results ever reported on these datasets.本文具体的贡献如下：我们在ILSVRC-2010和ILSVRC-2012[2]的ImageNet子集上训练了到目前为止最大的神经网络之一，并取得了迄今为止在这些数据集上报道过的最好结果。
  • Starting in 2010, as part of the Pascal Visual Object Challenge, an annual competition called the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) has been held.从2010年起，作为Pascal视觉对象挑战赛的一部分，每年都会举办ImageNet大规模视觉识别挑战赛（ILSVRC）。
  • ILSVRC uses a subset of ImageNet with roughly 1000 images in each of 1000 categories.ILSVRC使用ImageNet的一个子集，1000个类别每个类别大约1000张图像。
  • ILSVRC-2010 is the only version of ILSVRC for which the test set labels are available, so this is the version on which we performed most of our experiments.ILSVRC-2010是ILSVRC竞赛中唯一可以获得测试集标签的版本，因此我们大多数实验都是在这个版本上运行的。
  • ILSVRC-2010 is the only version of ILSVRC for which the test set labels are available, so this is the version on which we performed most of our experiments.ILSVRC-2010是ILSVRC竞赛中唯一可以获得测试集标签的版本，因此我们大多数实验都是在这个版本上运行的。
  • Since we also entered our model in the ILSVRC-2012 competition, in Section 6 we report our results on this version of the dataset as well, for which test set labels are unavailable.由于我们也使用我们的模型参加了ILSVRC-2012竞赛，因此在第六节我们也报告了模型在这个版本的数据集上的结果，这个版本的测试标签是不可获得的。
  • Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label, this turns out to be insufficient to learn so many parameters without considerable overfitting.尽管ILSVRC的1000类使每个训练样本从图像到标签的映射上强加了10比特的约束，但这不足以学习这么多的参数而没有相当大的过拟合。
  • Our results on ILSVRC-2010 are summarized in Table 1.我们在ILSVRC-2010上的结果概括为表1。
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24].在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
  • Table 1: Comparison of results on ILSVRC-2010 test set.表1：ILSVRC-2010测试集上的结果对比。
  • We also entered our model in the ILSVRC-2012 competition and report our results in Table 2.我们也用我们的模型参加了ILSVRC-2012竞赛并在表2中报告了我们的结果。
  • Since the ILSVRC-2012 test set labels are not publicly available, we cannot report test error rates for all the models that we tried.由于ILSVRC-2012的测试集标签不可以公开得到，我们不能报告我们尝试的所有模型的测试错误率。
  • Training one CNN, with an extra sixth convolutional layer over the last pooling layer, to classify the entire ImageNet Fall 2011 release (15M images, 22K categories), and then “fine-tuning” it on ILSVRC-2012 gives an error rate of 16.6%.为了对ImageNet 2011秋季发布的整个数据集（1500万图像，22000个类别）进行分类，我们在最后的池化层之后有一个额外的第6卷积层，训练了一个CNN，然后在它上面进行“fine-tuning”，在ILSVRC-2012取得了16.6%的错误率。
  • Table 2: Comparison of error rates on ILSVRC-2012 validation and test sets.表2：ILSVRC-2012验证集和测试集的误差对比。
  • Figure 4: (Left) Eight ILSVRC-2010 test images and the five labels considered most probable by our model.图4：（左）8张ILSVRC-2010测试图像和我们的模型认为最可能的5个标签。
  • (Right) Five ILSVRC-2010 test images in the first column.（右）第一列是5张ILSVRC-2010测试图像。
2 nonlinearity
(7) 		[nɒnlɪnɪ'ærɪtɪ]
  • 3.1 ReLU Nonlinearity3.1 ReLU非线性
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$.考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$.考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs).根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
  • For example, Jarrett et al. [11] claim that the nonlinearity $f(x) = \lvert \tanh(x) \rvert$ works particularly well with their type of contrast normalization followed by local average pooling on the Caltech-101 dataset.例如，Jarrett等人[11]声称非线性函数$f(x) = \lvert \tanh(x) \rvert$与其对比度归一化一起，然后是局部均值池化，在Caltech-101数据集上工作的非常好。
  • Denoting by $a_{x,y}^i$ the activity of a neuron computed by applying kernel $i$ at position $(x, y)$ and then applying the ReLU nonlinearity, the response-normalized activity $b^i_{x,y}$ is given by the expression$a_{x,y}^i$表示神经元激活，通过在$(x, y)$位置应用核i，然后应用ReLU非线性来计算，响应归一化激活$b^i_{x,y}$通过下式给定：
  • We applied this normalization after applying the ReLU nonlinearity in certain layers (see Section 3.5).我们在特定的层使用的ReLU非线性之后应用了这种归一化（请看3.5小节）。
3 augmentation
(5) 		[ˌɔ:ɡmen'teɪʃn]
  • 4.1 Data Augmentation4.1 数据增强
  • We employ two distinct forms of data augmentation, both of which allow transformed images to be produced from the original images with very little computation, so the transformed images do not need to be stored on disk.我们使用了两种独特的数据增强方式，这两种方式都可以从原始图像通过非常少的计算量产生变换的图像，因此变换图像不需要存储在硬盘上。
  • So these data augmentation schemes are, in effect, computationally free.因此，实际上这些数据增强方案是计算免费的。
  • The first form of data augmentation consists of generating image translations and horizontal reflections.第一种数据增强方式包括产生图像变换和水平翻转。
  • The second form of data augmentation consists of altering the intensities of the RGB channels in training images.第二种数据增强方式包括改变训练图像的RGB通道的强度。
4 e.g.
(4) 		[ˌi: ˈdʒi:]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations.直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • And indeed, the shortcomings of small image datasets have been widely recognized (e.g., Pinto et al. [21]), but it has only recently become possible to collect labeled datasets with millions of images.实际上，小图像数据集的缺点已经被广泛认识到（例如，Pinto et al. [21]），但收集上百万图像的标注数据仅在最近才变得的可能。
  • Traditionally, the neighborhoods summarized by adjacent pooling units do not overlap (e.g., [17, 11, 4]).习惯上，相邻池化单元归纳的区域是不重叠的（例如[17, 11, 4]）。
  • The easiest and most common method to reduce overfitting on image data is to artificially enlarge the dataset using label-preserving transformations (e.g., [25, 4, 5]).图像数据上最简单常用的用来减少过拟合的方法是使用标签保留变换（例如[25, 4, 5]）来人工增大数据集。
5 CIFAR
(4) 		[!≈ si: aɪ ef eɪ ɑ:(r)]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations.直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • This is demonstrated in Figure 1, which shows the number of iterations required to reach 25% training error on the CIFAR-10 dataset for a particular four-layer convolutional network.在图1中，对于一个特定的四层卷积网络，在CIFAR-10数据集上达到25%的训练误差所需要的迭代次数可以证实这一点。
  • Figure 1: A four-layer convolutional neural network with ReLUs (solid line) reaches a 25% training error rate on CIFAR-10 six times faster than an equivalent network with tanh neurons (dashed line).图1：使用ReLU的四层卷积神经网络在CIFAR-10数据集上达到25%的训练误差比使用tanh神经元的等价网络（虚线）快六倍。
  • We also verified the effectiveness of this scheme on the CIFAR-10 dataset: a four-layer CNN achieved a 13% test error rate without normalization and 11% with normalization.我们也在CIFAR-10数据集上验证了这个方案的有效性：一个乜嘢归一化的四层CNN取得了13%的错误率，而使用归一化取得了11%的错误率。
6 saturate
(4) 		[ˈsætʃəreɪt]
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$.考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • This plot shows that we would not have been able to experiment with such large neural networks for this work if we had used traditional saturating neuron models.这幅图表明，如果我们采用传统的饱和神经元模型，我们将不能在如此大的神经网络上实验该工作。
  • The magnitude of the effect demonstrated here varies with network architecture, but networks with ReLUs consistently learn several times faster than equivalents with saturating neurons.影响的大小随着网络结构的变化而变化，这一点已得到证实，但使用ReLU的网络都比等价的饱和神经元快几倍。
  • ReLUs have the desirable property that they do not require input normalization to prevent them from saturating.ReLU具有让人满意的特性，它不需要通过输入归一化来防止饱和。
7 GTX
(3) 		[!≈ dʒi: ti: eks]
  • Our network takes between five and six days to train on two GTX 580 3GB GPUs.我们的网络在两个GTX 580 3GB GPU上训练五六天。
  • A single GTX 580 GPU has only 3GB of memory, which limits the maximum size of the networks that can be trained on it.单个GTX580 GPU只有3G内存，这限制了可以在GTX580上进行训练的网络最大尺寸。
  • We trained the network for roughly 90 cycles through the training set of 1.2 million images, which took five to six days on two NVIDIA GTX 580 3GB GPUs.我们在120万图像的训练数据集上训练神经网络大约90个循环，在两个NVIDIA GTX 580 3GB GPU上花费了五到六天。
8 Euclidean
(3) 		[ju:ˈklidiən]
  • The remaining columns show the six training images that produce feature vectors in the last hidden layer with the smallest Euclidean distance from the feature vector for the test image.剩下的列展示了6张训练图像，这些图像在最后的隐藏层的特征向量与测试图像的特征向量有最小的欧氏距离。
  • If two images produce feature activation vectors with a small Euclidean separation, we can say that the higher levels of the neural network consider them to be similar.如果两幅图像生成的特征激活向量之间有较小的欧式距离，我们可以认为神经网络的更高层特征认为它们是相似的。
  • Computing similarity by using Euclidean distance between two 4096-dimensional, real-valued vectors is inefficient, but it could be made efficient by training an auto-encoder to compress these vectors to short binary codes.通过两个4096维实值向量间的欧氏距离来计算相似性是效率低下的，但通过训练一个自动编码器将这些向量压缩为短二值编码可以使其变得高效。
9 retrieve
(3) 		[rɪˈtri:v]
  • Notice that at the pixel level, the retrieved training images are generally not close in L2 to the query images in the first column.注意在像素级别，检索到的训练图像与第一列的查询图像在L2上通常是不接近的。
  • For example, the retrieved dogs and elephants appear in a variety of poses.例如，检索的狗和大象似乎有很多姿态。
  • This should produce a much better image retrieval method than applying auto-encoders to the raw pixels [14], which does not make use of image labels and hence has a tendency to retrieve images with similar patterns of edges, whether or not they are semantically similar.这应该会产生一种比将自动编码器应用到原始像素上[14]更好的图像检索方法，自动编码器应用到原始像素上的方法没有使用图像标签，因此会趋向于检索与要检索的图像具有相似边缘模式的图像，无论它们是否是语义上相似。
10 non-saturating
(2) 		[!≈ nɒn ˈsætʃəreitɪŋ]
  • To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation.为了训练的更快，我们使用了非饱和神经元并对卷积操作进行了非常有效的GPU实现。
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$.考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
11 Caltech
(2) 		['kæltek]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations.直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • For example, Jarrett et al. [11] claim that the nonlinearity $f(x) = \lvert \tanh(x) \rvert$ works particularly well with their type of contrast normalization followed by local average pooling on the Caltech-101 dataset.例如，Jarrett等人[11]声称非线性函数$f(x) = \lvert \tanh(x) \rvert$与其对比度归一化一起，然后是局部均值池化，在Caltech-101数据集上工作的非常好。
12 label-preserving
(2) 		[!≈ ˈleɪbl prɪ'zɜ:vɪŋ]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations.直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
  • The easiest and most common method to reduce overfitting on image data is to artificially enlarge the dataset using label-preserving transformations (e.g., [25, 4, 5]).图像数据上最简单常用的用来减少过拟合的方法是使用标签保留变换（例如[25, 4, 5]）来人工增大数据集。
13 descent
(2) 		[dɪˈsent]
  • In terms of training time with gradient descent, these saturating nonlinearities are much slower than the non-saturating nonlinearity $f(x) = \max(0,x)$.考虑到梯度下降的训练时间，这些饱和的非线性比非饱和非线性$f(x) = \max(0,x)$更慢。
  • We trained our models using stochastic gradient descent with a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005.我们使用随机梯度下降来训练我们的模型，样本的batch size为128，动量为0.9，权重衰减为0.0005。
14 Jarrett
(2)
  • For example, Jarrett et al. [11] claim that the nonlinearity $f(x) = \lvert \tanh(x) \rvert$ works particularly well with their type of contrast normalization followed by local average pooling on the Caltech-101 dataset.例如，Jarrett等人[11]声称非线性函数$f(x) = \lvert \tanh(x) \rvert$与其对比度归一化一起，然后是局部均值池化，在Caltech-101数据集上工作的非常好。
  • This scheme bears some resemblance to the local contrast normalization scheme of Jarrett et al. [11], but ours would be more correctly termed “brightness normalization”, since we do not subtract the mean activity.这个方案与Jarrett等人[11]的局部对比度归一化方案有一定的相似性，但我们更恰当的称其为“亮度归一化”，因此我们没有减去均值。
15 parallelization
(2) 		[pærəlɪlaɪ'zeɪʃn]
  • Current GPUs are particularly well-suited to cross-GPU parallelization, as they are able to read from and write to one another’s memory directly, without going through host machine memory.目前的GPU非常适合跨GPU并行，因为它们可以直接互相读写内存，而不需要通过主机内存。
  • The parallelization scheme that we employ essentially puts half of the kernels (or neurons) on each GPU, with one additional trick: the GPUs communicate only in certain layers.我们采用的并行方案基本上每个GPU放置一半的核（或神经元），还有一个额外的技巧：只在某些特定的层上进行GPU通信。
16 connectivity
(2) 		[ˌkɒnekˈtɪvɪti]
  • Choosing the pattern of connectivity is a problem for cross-validation, but this allows us to precisely tune the amount of communication until it is an acceptable fraction of the amount of computation.连接模式的选择是一个交叉验证问题，但这可以让我们准确地调整通信数量，直到它的计算量在可接受的范围内。
  • Notice the specialization exhibited by the two GPUs, a result of the restricted connectivity described in Section 3.5.注意两个GPU表现出的专业化，3.5小节中描述的受限连接的结果。
17 layer-part
(2) 		[!≈ ˈleɪə(r) pɑ:t]
  • One GPU runs the layer-parts at the top of the figure while the other runs the layer-parts at the bottom.在图的顶部，一个GPU运行在部分层上，而在图的底部，另一个GPU运行在部分层上。
  • One GPU runs the layer-parts at the top of the figure while the other runs the layer-parts at the bottom.在图的顶部，一个GPU运行在部分层上，而在图的底部，另一个GPU运行在部分层上。
18 response-normalized
(2) 		[!≈ rɪˈspɒns 'nɔ:məlaɪzd]
  • Denoting by $a_{x,y}^i$ the activity of a neuron computed by applying kernel $i$ at position $(x, y)$ and then applying the ReLU nonlinearity, the response-normalized activity $b^i_{x,y}$ is given by the expression$a_{x,y}^i$表示神经元激活，通过在$(x, y)$位置应用核i，然后应用ReLU非线性来计算，响应归一化激活$b^i_{x,y}$通过下式给定：
  • The second convolutional layer takes as input the (response-normalized and pooled) output of the first convolutional layer and filters it with 256 kernels of size 5 × 5 × 48.第2卷积层使用用第1卷积层的输出（响应归一化和池化）作为输入，并使用256个核进行滤波，核大小为5 × 5 × 48。
19 neighboring
(2) 		['neɪbərɪŋ]
  • Pooling layers in CNNs summarize the outputs of neighboring groups of neurons in the same kernel map.CNN中的池化层归纳了同一核映射上相邻组神经元的输出。
  • The first convolutional layer filters the 224 × 224 × 3 input image with 96 kernels of size 11 × 11 × 3 with a stride of 4 pixels (this is the distance between the receptive field centers of neighboring neurons in a kernel map).第1卷积层使用96个核对224 × 224 × 3的输入图像进行滤波，核大小为11 × 11 × 3，步长是4个像素（核映射中相邻神经元感受野中心之间的距离）。
20 response-normalization
(2) 		[!≈ rɪˈspɒns ˌnɔ:məlaɪ'zeɪʃn]
  • Response-normalization layers follow the first and second convolutional layers.第1，2卷积层之后是响应归一化层。
  • Max-pooling layers, of the kind described in Section 3.4, follow both response-normalization layers as well as the fifth convolutional layer.3.4节描述的这种最大池化层在响应归一化层和第5卷积层之后。
21 eigenvalue
(2) 		['aɪdʒənˌvælju:]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1.对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable.$p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
22 Gaussian
(2) 		['gaʊsɪən]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1.对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
  • We initialized the weights in each layer from a zero-mean Gaussian distribution with standard deviation 0.01.我们使用均值为0，标准差为0.01的高斯分布对每一层的权重进行初始化。
23 deviation
(2) 		[ˌdi:viˈeɪʃn]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1.对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
  • We initialized the weights in each layer from a zero-mean Gaussian distribution with standard deviation 0.01.我们使用均值为0，标准差为0.01的高斯分布对每一层的权重进行初始化。
24 ith
(2)
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable.$p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
  • where $i$ is the iteration index, $v$ is the momentum variable, $\varepsilon$ is the learning rate, and $\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$ is the average over the ith batch $D_i$ of the derivative of the objective with respect to $w$, evaluated at $w_i$.$i$是迭代索引，$v$是动量变量，$\varepsilon$是学习率，$\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$是目标函数对$w$，在$w_i$上的第$i$批微分$D_i$的平均。
25 aforementioned
(2) 		[əˌfɔ:ˈmenʃənd]
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable.$p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
  • Averaging the predictions of two CNNs that were pre-trained on the entire Fall 2011 release with the aforementioned five CNNs gives an error rate of 15.3%.对在ImageNet 2011秋季发布的整个数据集上预训练的两个CNN和前面提到的五个CNN的预测进行平均得到了15.3%的错误率。
26 momentum
(2) 		[məˈmentəm]
  • We trained our models using stochastic gradient descent with a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005.我们使用随机梯度下降来训练我们的模型，样本的batch size为128，动量为0.9，权重衰减为0.0005。
  • where $i$ is the iteration index, $v$ is the momentum variable, $\varepsilon$ is the learning rate, and $\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$ is the average over the ith batch $D_i$ of the derivative of the objective with respect to $w$, evaluated at $w_i$.$i$是迭代索引，$v$是动量变量，$\varepsilon$是学习率，$\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$是目标函数对$w$，在$w_i$上的第$i$批微分$D_i$的平均。
27 FVs
(2)
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24].在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
  • The second-best contest entry achieved an error rate of 26.2% with an approach that averages the predictions of several classifiers trained on FVs computed from different types of densely-sampled features [7].第二名的最好竞赛输入取得了26.2%的错误率，他的方法是对FV上训练的一些分类器的预测结果进行平均，FV在不同类型密集采样特征计算得到的。
28 densely-sampled
(2) 		[!≈ denslɪ 'sɑ:mpld]
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24].在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
  • The second-best contest entry achieved an error rate of 26.2% with an approach that averages the predictions of several classifiers trained on FVs computed from different types of densely-sampled features [7].第二名的最好竞赛输入取得了26.2%的错误率，他的方法是对FV上训练的一些分类器的预测结果进行平均，FV在不同类型密集采样特征计算得到的。
29 italic
(2) 		[ɪˈtælɪk]
  • In italics are best results achieved by others.斜体是其它人取得的最好结果。
  • In italics are best results achieved by others.斜线部分是其它人取得的最好的结果。
30 specialization
(2) 		[ˌspeʃəlaɪ'zeɪʃn]
  • Notice the specialization exhibited by the two GPUs, a result of the restricted connectivity described in Section 3.5.注意两个GPU表现出的专业化，3.5小节中描述的受限连接的结果。
  • This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs).这种专业化在每次运行时都会发生，并且是与任何特别的随机权重初始化（以GPU的重新编号为模）无关的。
31 LSVRC
(1) 		[!≈ el es vi: ɑ:(r) si:]
  • We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes.我们训练了一个大型深度卷积神经网络来将ImageNet LSVRC-2010竞赛的120万高分辨率的图像分到1000不同的类别中。
32 variant
(1) 		[ˈveəriənt]
  • We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.我们也使用这个模型的一个变种参加了ILSVRC-2012竞赛，赢得了冠军并且与第二名 top-5 26.2%的错误率相比，我们取得了top-5 15.3%的错误率。
33 NORB
(1) 		[!≈ en əu ɑ:(r) bi:]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations.直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
34 augment
(1) 		[ɔ:gˈment]
  • Until recently, datasets of labeled images were relatively small -- on the order of tens of thousands of images (e.g., NORB [16], Caltech-101/256 [8, 9], and CIFAR-10/100 [12]). Simple recognition tasks can be solved quite well with datasets of this size, especially if they are augmented with label-preserving transformations.直到最近，标注图像的数据集都相对较小--在几万张图像的数量级上（例如，NORB[16]，Caltech-101/256 [8, 9]和CIFAR-10/100 [12]）。简单的识别任务在这样大小的数据集上可以被解决的相当好，尤其是如果通过标签保留变换进行数据增强的情况下。
35 MNIST
(1) 		[!≈ em en aɪ es ti:]
  • For example, the current best error rate on the MNIST digit-recognition task (<0.3%) approaches human performance [4].例如，目前在MNIST数字识别任务上（<0.3%）的最好准确率已经接近了人类水平[4]。
36 variability
(1) 		[ˌveəriəˈbɪləti]
  • But objects in realistic settings exhibit considerable variability, so to learn to recognize them it is necessary to use much larger training sets.但真实环境中的对象表现出了相当大的可变性，因此为了学习识别它们，有必要使用更大的训练数据集。
37 Pinto
(1) 		[ˈpɪntəʊ]
  • And indeed, the shortcomings of small image datasets have been widely recognized (e.g., Pinto et al. [21]), but it has only recently become possible to collect labeled datasets with millions of images.实际上，小图像数据集的缺点已经被广泛认识到（例如，Pinto et al. [21]），但收集上百万图像的标注数据仅在最近才变得的可能。
38 LabelMe
(1)
  • The new larger datasets include LabelMe [23], which consists of hundreds of thousands of fully-segmented images, and ImageNet [6], which consists of over 15 million labeled high-resolution images in over 22,000 categories.新的更大的数据集包括LabelMe [23]，它包含了数十万张完全分割的图像，ImageNet[6]，它包含了22000个类别上的超过1500万张标注的高分辨率的图像。
39 immense
(1) 		[ɪˈmens]
  • However, the immense complexity of the object recognition task means that this problem cannot be specified even by a dataset as large as ImageNet, so our model should also have lots of prior knowledge to compensate for all the data we don’t have.然而对象识别任务的巨大复杂性意味着这个问题不能被指定，即使通过像ImageNet这样的大数据集，因此我们的模型应该也有许多先验知识来补偿我们所没有的数据。
40 constitute
(1) 		[ˈkɒnstɪtju:t]
  • Convolutional neural networks (CNNs) constitute one such class of models [16, 11, 13, 18, 15, 22, 26].卷积神经网络(CNNs)构成了一个这样的模型[16, 11, 13, 18, 15, 22, 26]。
41 stationarity
(1) 		[steɪʃə'nærɪtɪ]
  • Their capacity can be controlled by varying their depth and breadth, and they also make strong and mostly correct assumptions about the nature of images (namely, stationarity of statistics and locality of pixel dependencies).它们的能力可以通过改变它们的广度和深度来控制，它们也可以对图像的本质进行强大且通常正确的假设（也就是说，统计的稳定性和像素依赖的局部性）。
42 feedforward
(1) 		[fi:d'fɔ:wəd]
  • Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.因此，与具有层次大小相似的标准前馈神经网络，CNNs有更少的连接和参数，因此它们更容易训练，而它们理论上的最佳性能可能仅比标准前馈神经网络差一点。
43 similarly-sized
(1) 		[!≈ ˈsɪmələli 'saɪzd]
  • Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.因此，与具有层次大小相似的标准前馈神经网络，CNNs有更少的连接和参数，因此它们更容易训练，而它们理论上的最佳性能可能仅比标准前馈神经网络差一点。
44 theoretically-best
(1) 		[!≈ ˌθɪə'retɪklɪ best]
  • Thus, compared to standard feedforward neural networks with similarly-sized layers, CNNs have much fewer connections and parameters and so they are easier to train, while their theoretically-best performance is likely to be only slightly worse.因此，与具有层次大小相似的标准前馈神经网络，CNNs有更少的连接和参数，因此它们更容易训练，而它们理论上的最佳性能可能仅比标准前馈神经网络差一点。
45 prohibitively
(1) 		[prəʊˈhɪbɪtɪvlɪ]
  • Despite the attractive qualities of CNNs, and despite the relative efficiency of their local architecture, they have still been prohibitively expensive to apply in large scale to high-resolution images.尽管CNN具有引人注目的特性，尽管CNN的局部架构相当有效，但将它们大规模的应用到到高分辨率图像中仍然是代价极高的。
46 interestingly-large
(1) 		[!≈ 'intristiŋli lɑ:dʒ]
  • Luckily, current GPUs, paired with a highly-optimized implementation of 2D convolution, are powerful enough to facilitate the training of interestingly-large CNNs, and recent datasets such as ImageNet contain enough labeled examples to train such models without severe overfitting.幸运的是，目前的GPU，搭配了高度优化的2D卷积实现，强大到足够促进有趣地大量CNN的训练，最近的数据集例如ImageNet包含足够的标注样本来训练这样的模型而没有严重的过拟合。
47 labelers
(1) 		[!≈ 'leɪblərs]
  • The images were collected from the web and labeled by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool.这些图像是从网上收集的，使用了Amazon’s Mechanical Turk的众包工具通过人工标注的。
48 Turk
(1) 		[tә:k]
  • The images were collected from the web and labeled by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool.这些图像是从网上收集的，使用了Amazon’s Mechanical Turk的众包工具通过人工标注的。
49 crowd-sourcing
(1) 		[!≈ kraʊd 'saʊəsɪŋ]
  • The images were collected from the web and labeled by human labelers using Amazon’s Mechanical Turk crowd-sourcing tool.这些图像是从网上收集的，使用了Amazon’s Mechanical Turk的众包工具通过人工标注的。
50 Pascal
(1) 		['pæskәl]
  • Starting in 2010, as part of the Pascal Visual Object Challenge, an annual competition called the ImageNet Large-Scale Visual Recognition Challenge (ILSVRC) has been held.从2010年起，作为Pascal视觉对象挑战赛的一部分，每年都会举办ImageNet大规模视觉识别挑战赛（ILSVRC）。
51 variable-resolution
(1) 		[!≈ ˈveəriəbl ˌrezəˈlu:ʃn]
  • ImageNet consists of variable-resolution images, while our system requires a constant input dimensionality.ImageNet包含各种分辨率的图像，而我们的系统要求不变的输入维度。
52 down-sampled
(1) 		[!≈ daʊn 'sɑ:mpld]
  • Therefore, we down-sampled the images to a fixed resolution of 256 × 256.因此，我们将图像进行下采样到固定的256×256分辨率。
53 rectangular
(1) 		[rek'tæŋɡjələ(r)]
  • Given a rectangular image, we first rescaled the image such that the shorter side was of length 256, and then cropped out the central 256×256 patch from the resulting image.给定一个矩形图像，我们首先缩放图像短边长度为256，然后从结果图像中裁剪中心的256×256大小的图像块。
54 rescale
(1) 		[ri:'skeɪl]
  • Given a rectangular image, we first rescaled the image such that the shorter side was of length 256, and then cropped out the central 256×256 patch from the resulting image.给定一个矩形图像，我们首先缩放图像短边长度为256，然后从结果图像中裁剪中心的256×256大小的图像块。
55 Nair
(1) 		['nɑ:ɪə]
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs).根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
56 Hinton
(1)
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs).根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
57 rectify
(1) 		[ˈrektɪfaɪ]
  • Following Nair and Hinton [20], we refer to neurons with this nonlinearity as Rectified Linear Units (ReLUs).根据Nair和Hinton[20]的说法，我们将这种非线性神经元称为修正线性单元(ReLU)。
58 dash
(1) 		[dæʃ]
  • Figure 1: A four-layer convolutional neural network with ReLUs (solid line) reaches a 25% training error rate on CIFAR-10 six times faster than an equivalent network with tanh neurons (dashed line).图1：使用ReLU的四层卷积神经网络在CIFAR-10数据集上达到25%的训练误差比使用tanh神经元的等价网络（虚线）快六倍。
59 consistently
(1) 		[kən'sɪstəntlɪ]
  • The magnitude of the effect demonstrated here varies with network architecture, but networks with ReLUs consistently learn several times faster than equivalents with saturating neurons.影响的大小随着网络结构的变化而变化，这一点已得到证实，但使用ReLU的网络都比等价的饱和神经元快几倍。
60 cross-GPU
(1)
  • Current GPUs are particularly well-suited to cross-GPU parallelization, as they are able to read from and write to one another’s memory directly, without going through host machine memory.目前的GPU非常适合跨GPU并行，因为它们可以直接互相读写内存，而不需要通过主机内存。
61 cross-validation
(1) 		[k'rɒsvælɪd'eɪʃn]
  • Choosing the pattern of connectivity is a problem for cross-validation, but this allows us to precisely tune the amount of communication until it is an acceptable fraction of the amount of computation.连接模式的选择是一个交叉验证问题，但这可以让我们准确地调整通信数量，直到它的计算量在可接受的范围内。
62 resultant
(1) 		[rɪˈzʌltənt]
  • The resultant architecture is somewhat similar to that of the “columnar” CNN employed by Ciresan et al. [5], except that our columns are not independent (see Figure 2).除了我们的列不是独立的之外（看图2），最终的架构有点类似于Ciresan等人[5]采用的“columnar” CNN。
63 columnar
(1) 		[kə'lʌmnə]
  • The resultant architecture is somewhat similar to that of the “columnar” CNN employed by Ciresan et al. [5], except that our columns are not independent (see Figure 2).除了我们的列不是独立的之外（看图2），最终的架构有点类似于Ciresan等人[5]采用的“columnar” CNN。
64 Ciresan
(1)
  • The resultant architecture is somewhat similar to that of the “columnar” CNN employed by Ciresan et al. [5], except that our columns are not independent (see Figure 2).除了我们的列不是独立的之外（看图2），最终的架构有点类似于Ciresan等人[5]采用的“columnar” CNN。
65 delineation
(1) 		[dɪˌlɪnɪ'eɪʃn]
  • Figure 2: An illustration of the architecture of our CNN, explicitly showing the delineation of responsibilities between the two GPUs.图 2：我们CNN架构图解，明确描述了两个GPU之间的责任。
66 arbitrary
(1) 		[ˈɑ:bɪtrəri]
  • The ordering of the kernel maps is of course arbitrary and determined before training begins.核映射的顺序当然是任意的，在训练开始前确定。
67 lateral
(1) 		[ˈlætərəl]
  • This sort of response normalization implements a form of lateral inhibition inspired by the type found in real neurons, creating competition for big activities amongst neuron outputs computed using different kernels.响应归一化的顺序实现了一种侧抑制形式，灵感来自于真实神经元中发现的类型，为使用不同核进行神经元输出计算的较大活动创造了竞争。
68 inhibition
(1) 		[ˌɪnhɪˈbɪʃn]
  • This sort of response normalization implements a form of lateral inhibition inspired by the type found in real neurons, creating competition for big activities amongst neuron outputs computed using different kernels.响应归一化的顺序实现了一种侧抑制形式，灵感来自于真实神经元中发现的类型，为使用不同核进行神经元输出计算的较大活动创造了竞争。
69 brightness
(1) 		['braɪtnəs]
  • This scheme bears some resemblance to the local contrast normalization scheme of Jarrett et al. [11], but ours would be more correctly termed “brightness normalization”, since we do not subtract the mean activity.这个方案与Jarrett等人[11]的局部对比度归一化方案有一定的相似性，但我们更恰当的称其为“亮度归一化”，因此我们没有减去均值。
70 overfit
(1)
  • We generally observe during training that models with overlapping pooling find it slightly more difficult to overfit.我们在训练过程中通常观察采用重叠池化的模型，发现它更难过拟合。
71 multinomial
(1) 		[ˌmʌltɪ'nəʊmɪəl]
  • Our network maximizes the multinomial logistic regression objective, which is equivalent to maximizing the average across training cases of the log-probability of the correct label under the prediction distribution.我们的网络最大化多项逻辑回归的目标，这等价于最大化预测分布下训练样本正确标签的对数概率的均值。
72 log-probability
(1) 		[!≈ lɒg ˌprɒbəˈbɪləti]
  • Our network maximizes the multinomial logistic regression objective, which is equivalent to maximizing the average across training cases of the log-probability of the correct label under the prediction distribution.我们的网络最大化多项逻辑回归的目标，这等价于最大化预测分布下训练样本正确标签的对数概率的均值。
73 non-linearity
(1) 		['nɒnlaɪn'ərɪtɪ]
  • The ReLU non-linearity is applied to the output of every convolutional and fully-connected layer.ReLU非线性应用在每个卷积层和全连接层的输出上。
74 receptive
(1) 		[rɪˈseptɪv]
  • The first convolutional layer filters the 224 × 224 × 3 input image with 96 kernels of size 11 × 11 × 3 with a stride of 4 pixels (this is the distance between the receptive field centers of neighboring neurons in a kernel map).第1卷积层使用96个核对224 × 224 × 3的输入图像进行滤波，核大小为11 × 11 × 3，步长是4个像素（核映射中相邻神经元感受野中心之间的距离）。
75 normalize
(1) 		[ˈnɔ:məlaɪz]
  • The third convolutional layer has 384 kernels of size 3 × 3 × 256 connected to the (normalized, pooled) outputs of the second convolutional layer.第3卷积层有384个核，核大小为3 × 3 × 256，与第2卷积层的输出（归一化的，池化的）相连。
76 bits
(1) 		['bɪts]
  • Although the 1000 classes of ILSVRC make each training example impose 10 bits of constraint on the mapping from image to label, this turns out to be insufficient to learn so many parameters without considerable overfitting.尽管ILSVRC的1000类使每个训练样本从图像到标签的映射上强加了10比特的约束，但这不足以学习这么多的参数而没有相当大的过拟合。
77 artificially
(1) 		[ˌɑ:tɪ'fɪʃəlɪ]
  • The easiest and most common method to reduce overfitting on image data is to artificially enlarge the dataset using label-preserving transformations (e.g., [25, 4, 5]).图像数据上最简单常用的用来减少过拟合的方法是使用标签保留变换（例如[25, 4, 5]）来人工增大数据集。
78 computationally
(1) 		[!≈ ˌkɒmpjuˈteɪʃənli]
  • So these data augmentation schemes are, in effect, computationally free.因此，实际上这些数据增强方案是计算免费的。
79 interdependent
(1) 		[ˌɪntədɪˈpendənt]
  • This increases the size of our training set by a factor of 2048, though the resulting training examples are, of course, highly interdependent.这通过一个2048因子增大了我们的训练集，尽管最终的训练样本是高度相关的。
80 PCA
(1) 		[!≈ pi: si: eɪ]
  • Specifically, we perform PCA on the set of RGB pixel values throughout the ImageNet training set.具体地，我们在整个ImageNet训练集上对RGB像素值集合执行PCA。
81 proportional
(1) 		[prəˈpɔ:ʃənl]
  • To each training image, we add multiples of the found principal components, with magnitudes proportional to the corresponding eigenvalues times a random variable drawn from a Gaussian with mean zero and standard deviation 0.1.对于每幅训练图像，我们加上多倍找到的主成分，大小成正比的对应特征值乘以一个随机变量，随机变量通过均值为0，标准差为0.1的高斯分布得到。
82 eigenvector
(1) 		['aɪgənvektə(r)]
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable.$p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
83 covariance
(1) 		[kəʊ'veərɪəns]
  • where $p_i$ and $\lambda_i$ are ith eigenvector and eigenvalue of the $3 × 3$ covariance matrix of RGB pixel values, respectively, and $\alpha_i$ is the aforementioned random variable.$p_i$，$\lambda_i$分别是RGB像素值$3 × 3$协方差矩阵的第i个特征向量和特征值，$\alpha_i$是前面提到的随机变量。
84 invariant
(1) 		[ɪnˈveəriənt]
  • This scheme approximately captures an important property of natural images, namely, that object identity is invariant to changes in the intensity and color of the illumination.这个方案近似抓住了自然图像的一个重要特性，即光照的颜色和强度发生变化时，目标身份是不变的。
85 co-adaptation
(1) 		[!≈ kəʊ ˌædæpˈteɪʃn]
  • This technique reduces complex co-adaptations of neurons, since a neuron cannot rely on the presence of particular other neurons.这个技术减少了复杂的神经元互适应，因为一个神经元不能依赖特定的其它神经元的存在。
86 conjunction
(1) 		[kənˈdʒʌŋkʃn]
  • It is, therefore, forced to learn more robust features that are useful in conjunction with many different random subsets of the other neurons.因此，神经元被强迫学习更鲁棒的特征，它在与许多不同的其它神经元的随机子集结合时是有用的。
87 approximation
(1) 		[əˌprɒksɪˈmeɪʃn]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks.在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
88 geometric
(1) 		[ˌdʒi:əˈmetrɪk]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks.在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
89 predictive
(1) 		[prɪˈdɪktɪv]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks.在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
90 exponentially-many
(1) 		[!≈ ˌekspə'nenʃəlɪ ˈmeni]
  • At test time, we use all the neurons but multiply their outputs by 0.5, which is a reasonable approximation to taking the geometric mean of the predictive distributions produced by the exponentially-many dropout networks.在测试时，我们使用所有的神经元但它们的输出乘以0.5，对指数级的许多失活网络的预测分布进行几何平均，这是一种合理的近似。
91 stochastic
(1) 		[stə'kæstɪk]
  • We trained our models using stochastic gradient descent with a batch size of 128 examples, momentum of 0.9, and weight decay of 0.0005.我们使用随机梯度下降来训练我们的模型，样本的batch size为128，动量为0.9，权重衰减为0.0005。
92 regularizer
(1)
  • In other words, weight decay here is not merely a regularizer: it reduces the model’s training error.换句话说，权重衰减不仅仅是一个正则项：它减少了模型的训练误差。
93 derivative
(1) 		[dɪˈrɪvətɪv]
  • where $i$ is the iteration index, $v$ is the momentum variable, $\varepsilon$ is the learning rate, and $\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$ is the average over the ith batch $D_i$ of the derivative of the objective with respect to $w$, evaluated at $w_i$.$i$是迭代索引，$v$是动量变量，$\varepsilon$是学习率，$\langle \frac{\partial L} {\partial w} |_{w_i}\rangle _{D_i}$是目标函数对$w$，在$w_i$上的第$i$批微分$D_i$的平均。
94 heuristic
(1) 		[hjuˈrɪstɪk]
  • The heuristic which we followed was to divide the learning rate by 10 when the validation error rate stopped improving with the current learning rate.当验证误差在当前的学习率下停止提供时，我们遵循启发式的方法将学习率除以10。
95 termination
(1) 		[ˌtɜ:mɪˈneɪʃn]
  • The learning rate was initialized at 0.01 and reduced three times prior to termination.学习率初始化为0.01，在训练停止之前降低三次。
96 NVIDIA
(1) 		[ɪn'vɪdɪə]
  • We trained the network for roughly 90 cycles through the training set of 1.2 million images, which took five to six days on two NVIDIA GTX 580 3GB GPUs.我们在120万图像的训练数据集上训练神经网络大约90个循环，在两个NVIDIA GTX 580 3GB GPU上花费了五到六天。
97 sparse-coding
(1) 		[!≈ spɑ:s 'kəʊdɪŋ]
  • The best performance achieved during the ILSVRC-2010 competition was 47.1% and 28.2% with an approach that averages the predictions produced from six sparse-coding models trained on different features [2], and since then the best published results are 45.7% and 25.7% with an approach that averages the predictions of two classifiers trained on Fisher Vectors (FVs) computed from two types of densely-sampled features [24].在ILSVRC-2010竞赛中最佳结果是top-1 47.1%，top-5 28.2%，使用的方法是对6个在不同特征上训练的稀疏编码模型生成的预测进行平均，从那时起已公布的最好结果是top-1 45.7%，top-5 25.7%，使用的方法是平均在Fisher向量（FV）上训练的两个分类器的预测结果，Fisher向量是通过两种密集采样特征计算得到的[24]。
98 interchangeably
(1) 		[ɪntəˈtʃeɪndʒəblɪ]
  • In the remainder of this paragraph, we use validation and test error rates interchangeably because in our experience they do not differ by more than 0.1% (see Table 2).在这段的其余部分，我们会使用验证误差率和测试误差率互换，因为在我们的实验中它们的差别不会超过0.1%（看图2）。
99 asterisk
(1) 		[ˈæstərɪsk]
  • Models with an asterisk were “pre-trained” to classify the entire ImageNet 2011 Fall release.带星号的是“预训练的”对ImageNet 2011秋季数据集进行分类的模型。
100 convention
(1) 		[kənˈvenʃn]
  • On this dataset we follow the convention in the literature of using half of the images for training and half for testing.在这个数据集上我们按照惯例用一半的图像来训练，一半的图像来测试。
101 appreciably
(1) 		[ə'pri:ʃəblɪ]
  • Since there is no established test set, our split necessarily differs from the splits used by previous authors, but this does not affect the results appreciably.由于没有建立测试集，我们的数据集分割有必要不同于以前作者的数据集分割，但这对结果没有明显的影响。
102 Qualitative
(1) 		[ˈkwɒlɪtətɪv]
  • 6.1 Qualitative Evaluations6.1 定性评估
103 data-connected
(1) 		[!≈ ˈdeɪtə kə'nektɪd]
  • Figure 3 shows the convolutional kernels learned by the network’s two data-connected layers.图3显示了网络的两个数据连接层学习到的卷积核。
104 orientation-selective
(1) 		[!≈ ˌɔ:riənˈteɪʃn sɪˈlektɪv]
  • The network has learned a variety of frequency and orientation-selective kernels, as well as various colored blobs.网络学习到了大量的频率核、方向选择核，也学到了各种颜色点。
105 blob
(1) 		[blɒb]
  • The network has learned a variety of frequency and orientation-selective kernels, as well as various colored blobs.网络学习到了大量的频率核、方向选择核，也学到了各种颜色点。
106 color-agnostic
(1) 		[!≈ 'kʌlə(r) ægˈnɒstɪk]
  • The kernels on GPU 1 are largely color-agnostic, while the kernels on on GPU 2 are largely color-specific.GPU 1上的核主要是没有颜色的，而GPU 2上的核主要是针对颜色的。
107 color-specific
(1) 		[!≈ 'kʌlə(r) spəˈsɪfɪk]
  • The kernels on GPU 1 are largely color-agnostic, while the kernels on on GPU 2 are largely color-specific.GPU 1上的核主要是没有颜色的，而GPU 2上的核主要是针对颜色的。
108 modulo
(1) 		['mɒdjʊləʊ]
  • This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs).这种专业化在每次运行时都会发生，并且是与任何特别的随机权重初始化（以GPU的重新编号为模）无关的。
109 renumber
(1) 		['ri:'nʌmbə]
  • This kind of specialization occurs during every run and is independent of any particular random weight initialization (modulo a renumbering of the GPUs).这种专业化在每次运行时都会发生，并且是与任何特别的随机权重初始化（以GPU的重新编号为模）无关的。
110 qualitatively
(1) 		['kwɒlɪtətɪvlɪ]
  • In the left panel of Figure 4 we qualitatively assess what the network has learned by computing its top-5 predictions on eight test images.在图4的左边部分，我们通过在8张测试图像上计算它的top-5预测定性地评估了网络学习到的东西。
111 off-center
(1) 		['ɔ:fs'entə]
  • Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net.注意即使是不在图像中心的目标也能被网络识别，例如左上角的小虫。
112 mite
(1) 		[maɪt]
  • Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net.注意即使是不在图像中心的目标也能被网络识别，例如左上角的小虫。
113 top-left
(1) 		[!≈ tɒp left]
  • Notice that even off-center objects, such as the mite in the top-left, can be recognized by the net.注意即使是不在图像中心的目标也能被网络识别，例如左上角的小虫。
114 plausible
(1) 		[ˈplɔ:zəbl]
  • For example, only other types of cat are considered plausible labels for the leopard.例如，对于美洲豹来说，只有其它类型的猫被认为是看似合理的标签。
115 leopard
(1) 		[ˈlepəd]
  • For example, only other types of cat are considered plausible labels for the leopard.例如，对于美洲豹来说，只有其它类型的猫被认为是看似合理的标签。
116 grille
(1) 		[ɡrɪl]
  • In some cases (grille, cherry) there is genuine ambiguity about the intended focus of the photograph.在某些案例（格栅，樱桃）中，网络在意的图片焦点真的很含糊。
117 probe
(1) 		[prəʊb]
  • Another way to probe the network’s visual knowledge is to consider the feature activations induced by an image at the last, 4096-dimensional hidden layer.探索网络可视化知识的另一种方式是思考最后的4096维隐藏层在图像上得到的特征激活。
118 induce
(1) 		[ɪnˈdju:s]
  • Another way to probe the network’s visual knowledge is to consider the feature activations induced by an image at the last, 4096-dimensional hidden layer.探索网络可视化知识的另一种方式是思考最后的4096维隐藏层在图像上得到的特征激活。
119 supplementary
(1) 		[ˌsʌplɪˈmentri]
  • We present the results for many more test images in the supplementary material.我们在补充材料中对更多的测试图像呈现了这种结果。
120 real-valued
(1) 		['reɪɑ:lv'ælju:d]
  • Computing similarity by using Euclidean distance between two 4096-dimensional, real-valued vectors is inefficient, but it could be made efficient by training an auto-encoder to compress these vectors to short binary codes.通过两个4096维实值向量间的欧氏距离来计算相似性是效率低下的，但通过训练一个自动编码器将这些向量压缩为短二值编码可以使其变得高效。
121 retrieval
(1) 		[rɪˈtri:vl]
  • This should produce a much better image retrieval method than applying auto-encoders to the raw pixels [14], which does not make use of image labels and hence has a tendency to retrieve images with similar patterns of edges, whether or not they are semantically similar.这应该会产生一种比将自动编码器应用到原始像素上[14]更好的图像检索方法，自动编码器应用到原始像素上的方法没有使用图像标签，因此会趋向于检索与要检索的图像具有相似边缘模式的图像，无论它们是否是语义上相似。
122 semantically
(1) 		[sɪ'mæntɪklɪ]
  • This should produce a much better image retrieval method than applying auto-encoders to the raw pixels [14], which does not make use of image labels and hence has a tendency to retrieve images with similar patterns of edges, whether or not they are semantically similar.这应该会产生一种比将自动编码器应用到原始像素上[14]更好的图像检索方法，自动编码器应用到原始像素上的方法没有使用图像标签，因此会趋向于检索与要检索的图像具有相似边缘模式的图像，无论它们是否是语义上相似。
123 infero-temporal
(1)
  • Thus far, our results have improved as we have made our network larger and trained it longer but we still have many orders of magnitude to go in order to match the infero-temporal pathway of the human visual system.到目前为止，我们的结果已经提高了，因为我们的网络更大、训练时间更长，但为了匹配人类视觉系统的下颞线（视觉专业术语）我们仍然有许多数量级要达到。
124 pathway
(1) 		[ˈpɑ:θweɪ]
  • Thus far, our results have improved as we have made our network larger and trained it longer but we still have many orders of magnitude to go in order to match the infero-temporal pathway of the human visual system.到目前为止，我们的结果已经提高了，因为我们的网络更大、训练时间更长，但为了匹配人类视觉系统的下颞线（视觉专业术语）我们仍然有许多数量级要达到。